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Section I 


Why are we talking about validity? 

• Standards 
. NACAC 
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Validity Evidence 


"Validity refers to the degree to which 
evidence and theory support the 
interpretations of test scores entailed by 
proposed uses of tests." 


4 



Standards of Educational and Psychological Testing 
(AERA, APA, NOME, 1999) 
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Criterion-Related Validity 


• Criterion-related validity (also referred to as 
predictive validity) looks at the relationship 
between a test score and a desired outcome. 

• A criterion-related validation study is completed by 
collecting both the test scores that will be used and 
information on the criterion for the same students 

• SAT scores and first-year college GPA 

• SAT writing scores and English 101 grades 
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Validating a Test for a Particular Use 


• The most common approach used to validate an admission 
test for educational selection has been through the 
computation of validity coefficients and regression lines. 

• Validity coefficients are the computed correlation 
coefficients between predictor variables and a criterion or 
outcome variable(s), which can determine the predictive 
validity of a test. 

• A large correlation indicates strong predictive validity of a 
test to the criterion, however, a large correlation by itself 
does not satisfy all facets required of test validity. 
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Correlation 


Essentially, a correlation coefficient is a number 
between -1 and 1 which measures the degree to 
which two variables are linearly related. 


Strength (absolute magnitude) and direction (negative/positive) 
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Rule of Thumb for Interpreting Correlation 
Coefficients 


• A general rule of thumb for interpreting 
correlation coefficients is offered by Cohen 
( 1988 ): 

• small correlation has an absolute value of 
approximately 0.1 

• medium correlation has an absolute value of 
approximately 0.3 

• strong correlation has an absolute value of 
approximately 0.5 or higher 
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other Correlation Examples (nerdy indulgence) 
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FYGPA 


Another way to think of a correlation of 0.54 
Mean FYGPA by SAT Score Band 


4 
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3.6 

3.4 

3.2 
3 

2.8 

2.6 

2.4 

2.2 
2 



600-890 900-1190 1200-1490 1500-1790 1800-2090 2100-2400 


SAT SCORE BAND 
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Section II 


General ACES Information 
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Admitted Class Evaluation Service (ACES) 


• The Admitted Class Evaluation Service (ACES) is 
a free online service that predicts how admitted 
students will perform at a college or university 
generally (admission validity) and how successful 
students will be in specific courses (placement 
validity). 

http://professionals.colleaeboard.com/hiaher- 

ed/validity/aces 
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About ACES 


• ACES offers two models of validity studies: 

• Admission 

• Predictive 

• Placement 

• Predictive (ACCUPLACER, AP, SAT, SAT Subject Tests) 

• Concurrent (CLEP) 
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Admission Vaiidity Studies 


• The primary purpose of an admission validity study is to 
confirm (validate) or improve current measures used in 
admission decisions. 

• Will show how well admission criteria work alone and in 
combination with other predictors, and the most effective 
weighting for the predictors. 

• Success (the criterion) may be measured by college GPA and 
must be a continuous measure (not dichotomous) 

• Relevant predictors may be 

• SAT scores - Critical Reading, Math, or Writing 

• High school GPA, or Class Rank 

• Interview scores, and 

• Other information 
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Benefits of an ACES Admission Vaiidity Study 


• An ACES admission validity study will provide: 

• Information on the most useful predictors of success at 
an institution 

• Information allowing administrators to narrow the 
number of factors considered in the admission process 
without loss of predictive ability 

• Optimal equations for predicting the success of future 
students 

• A list of students at risk 

• A matched student-level data file for use in follow-up 
studies 
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Requesting an Admission Vaiidity Study 



• A minimum of 75 (complete) student records 
is required for an admission study. 


• Can specify up to 5 additional predictors - either 
from ACES-supplied data or from institutional data 
(provided that 75 or more students in your sample 
have that additional variable). 


• Results broken into three areas: gender, 
race/ethnicity, and best language spoken (provided 
that there are 75 or more students in the sample in 
at least 2 levels of the subgroup). 


• Can specify 2 additional subgroups - either ACES- 
supplied (e.g. degree-level goal, ability rating in 
math), from institutional data (e.g. resident versus 
commuter), or a combination. 
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Overview of ACES Process 


1 . 

2 . 

3 . 

4 . 

5 . 

6 . 

7 . 

8 . 


institutional contact/submitter will: 

Click link on ACES web site for a new ACES study request: 
https://cbweb1s.colleaeboard.org/aces/htnnl/newrvs.html 

Enter contact info (name, email, position, institution, etc.) 

Design study (choose predictors, subgroups, etc.) 

Receive automatically e-mailed user account, password, and 
request number from ACES 

Login to submit data at this site: 

https://cbweb1 s. colleaeboard.org/aces/html/submit1 .html 
Record all variable locations, indicate value labels, etc. 
Upload data file(s) 

ACES reports are returned to institutions 25 - 35 bus’ 


after the receipt of data. 
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Section III 


ACES Admission Validity Studies 
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ACES Web Site - Requesting a Study 


( CollegeBoard 
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Education Policy Si 
Advocacy 


College 

Guidance 


Membership Testing 
Educators - Information Sc Tools For Teachers. Counselors, Higher Education Faculh^ and 

Higher Ed Services 


ACES 


Recruitment 3c Admesions 


Financial AkJ 




MORE INFORMATION ABOUT ACES 




FactShe&t(.pdf/124K} 

Request an Admission Validity Study 
Request a PlacemerfrValidity Study 


Advising 3c Placerment 


Validity 

ACES™ 

Ad missio n Va IkJ ity Study + 

Placement Validity Study + 

Validity Handbook + 


ACES offers you free, easy-to-read validity studies 

ACES offers tv.ro types of validity studies— admission and placement. These 
studies identify the optimum combination of measures to predicta students future 
performance at your institution. ACES studies evaluate the differences for predicting 
the success of specific student groups and document the probabiiity of error. Each 
AGES report features: 

* In-depth analysis offindings 

* General background information to help you examine the study in greater 
detaii 

* Interpretive text highlighting key findings 



I K-12 Teacheir 


Fact Sheet C.pdf/124K} 

Request an Admission Validity Study 
Request a Placement Validity Study 

CONTACT 
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* Colorful presentation and graphics 

AGES admission validity studies typically use high school grade point average 
[HSGPA) or high school rank, along with SAT Reasoning Test™ scores, to establish 
the best combination of variables to predict student performance at your institution. 
You choose additional variables based on what you believe to be important 
contributors for predicting the academic success of your students. Examples of 
additional predictors are: 

* SAT SubjectTests™ scores 


Admitted Class Evaluation Service™ 
(ACES™) 

Research and Develop nrent 
The College Board 
45 Columbus Avenue 
New York, New York 1 0023-S992: 

Phone: (STT) SS5-2237 
FAX: (212} 649^27 

Questions about ACES online: 
a ces©in to . co llegeb o a rd . o rg . 


* Years of study in a particular subject area 
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ACES Web Site - Requesting a Study 

(contact information) 


a 


CollegeBoaz'd 

Admitted Glass Evaluation Service 


Admission Validity Study Request 


"Lse this form to submit a first request prior to submitting your institution's data. 

AH ACES Admission \"alidit>' Studies use first-year grade point average as the default criterion . You ha^'e the option of specifting a diflEerent criterion in Step 2 and of customizing other aspects 
of y our ^^alidit>' study in Step 3 . 

If at any time you have questions about the request process you may e-mail AC£S staff for assistanc.e. 

Coufact lufot'uiatiou 


(ISrote: Items marked by an " are required) 


Kame of institution: * 
Institution's College ^ 
Board code number: 

Last name 
First name 
M.I. 
Position Title 
E-mail address 
Telephone number 
Street 
Cit>^ 
State 
Zip code 
Department or school 
Secondary" contact 


Desigu Your Report 

Continue to Step 2 \-s"here you can specift" your criterion and predictors. 


Please enter this exactly as it is to appear in your fnal report. 
Institution code look-up: click here fPop-ups must be enabled to v;en’ this link) 


Clear Form 
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study Design 

Specify Criterion; Specify Predictors - HS measure 


Admission Validity Study Request (continued - 2 of 3) 

If at am' time you havo quostiom about the request process you may e-mail ACES staff for assistance. 

(Xote: Items marked by an ^ are required) 

Specify a Criteriou'^ 

I Select a criterion label ^ 

Specify Predictors 

All ACES Admission VaJidit>^ Studies use high school grade poiut average (GPA) or class rank. SAT Test scores and SAT Subject Test scores (optioual) as predictors. Please specfc 
belo\^^ your particular preferences for how these predictors should be used in your study. 


Specif' GPA or Class Rank 


All Admission Validit>^ Studies use either high school GPA or high school class rauk as predictors. Please indicate below which predictor you would prefer. You must also specfc if you will be 
providing this information or if you ’^^'ant to use ACES -supplied data. 


T^i>e of high school data 


Source of data 


r HSGPA 
r HSrank 


C from data supplied by your institution 
C from ACES -supplied data 
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Specify Predictors - SAT Scores 


Select the SAT Test(s) you wish to analyze as predictors. 

Please indicate the SAT Test score or scores that you would like to be included in your study by choosing one of the options below. Next, specify' whether you would like to use the highest or most 
recent score(s). The default choice is Critical Reading, Math, Writing. 

SAT Test: 


Individual Predictors 

Multiple Predictor Sets 

Single Scores 


^ Critical Reading 

Critical Reading, Math, Writing 

r Math 

[Critical Reading ~ Math], W nting 

Writing 

^ Critical Reading, \V*riting 

Composite Scores 

C Critical Reading, Math 

(sum of scores) 



f' Matii, W nting 

C [Critical Reading “ Math] 


[Critical Reading - Writing] 


^ [Math - Writing] 


[Critical Reading - Math - Writing] 


Specify- highest or most recent score(s) 

Use the highest score(s) 


^ Use the most recent score(s) 
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Specify Predictors - SAT Subject Tests 


Select the SAT Subject Tests you TrYish to analyze as predictors. 

\Mien SAT Subject Test scores are present in a student's record. Admission Validity" Studies routinely consider them as a predictor. There are as many as three options for using SAT Subject Test 
scores ^^ithin a single study. You may choose to use either scores from a particular SAT Subject Test(s), highest SAT Subject Test scores ^^ithin a student's record or average SAT Subject Test 
scores. Your choice should be based on the SAT Subject Test(s) that are used in making admission decisions at your institution. 

SAT Subject Tests: 


Using score(s) on specific SAT Subject Test(s) 

- OR- 

Using highest or average SAT Subject 
Tests 

Select first SAT Subject Test 

Select first SAT Subject Test predictor 

English 

Langnages 


C Highest iYon-Language SAT Subject Test 

U Literature 

U Chinese ’h^ith Listening 


C Highest of All SAT Subject Tests 


C French 


C A^'erage of H'^'o Highest Xon-Language 

Histoty & Social Sciences 

C French ’h^'ith Listening 


SAT Subject Tests 

U. S. Histoiy 

C German 


C Average of Two Highest SAT Subject Tests 

f World Histoiy^ 

C German \^ith Listening 




C Modem Hebrew 



iVIathemadcs 

C Italian 

- OR- 


^ Mathematics Le^^el 1 

C Japanese \Hth Listening 



^ Mathematics Le^^el 2 

C Korean ’^\ith Listening 




C Latin 



Science 

C Spanish 



^ Biolog>'- 

C Spanish ’^^ith Listening 



f Chemistrv^ 




r- 
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Specify Additional Predictors 


Specif^' Additiofiai 

You may also specif' as many as &'e additional pieces of information to be used as predictors. These can be dra^i^n either from ACES -supplied data or from 
information supplied and defined by your institution, providing that at least 75 students in your sample ha’^'e the additional ^■ariable(s) in their records. \"ariables 
to include would be those that are used in making admission decisions at your institution. 


I smg ACES-supplied data 


OR specif^^ your ov^fn; 


Usiog your data 

(all labels are limited to 20 characters) 


l~ Years of study in arts and music 
n Years of study in English 

n Years of study in foreign and classical languages 
l~ Years of study in mathematics 
n Years of study in natural sciences 
I Ycora of study in sociol sciences and history' 

l~ Xumber of different SAT Subject Tests 
I” Number of different AP Exams 
I” Number of Honors or AP courses 
I” Number of acthities during high school 

Desigu Your Report (cout.) 

Continue to Step 3 where you can specift^ additional subgroups and finalize the request process. 
Continue Clear Form 
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Examine Subgroups; Include Coursework 


Aduiissiou Valiclitj' Study Request (coutiuued - 3 of 3) 

If ai Lw VG u ato ui ihs rsqu^::i vi u mqi; s-mai! ACESjj^ for 

Speclh' Add Irion al Subgroups 

AH ACE S s tudi?s brsak dov^ii youi' result s on the basis of ge n d o’, ethnidtv, and n a tive La n gu a ge sp o ke n whsnevei- youi' sample indudes 7 5 
or more students for at least two kvds of a subfioup 75-males and 75-females) You may also spKit\'up to t^vo additional subgroups 
below^ a?aiii; using either ACE S - supplied data , your o^^ti data , or a combination (rs-: one subgroup from each) 


Select Additional Subgroup 1 


Select Ad d itio n a L S ub gro up 2 


r D?ne?-levdgoal 
^ Ability rating in nuth 
r Ability rating in science 
r Ability rating in ™ting 
r Othsr I 


r Deme- level goal 
^ Ability rating in nuth 
r Ability rating in science 
r Ability ratiiig ill ™ting 
r Othsi- I 


^^'ould you like result s for all Hispanic etlmic gr oup s to be aiulyzed and rep ort ed to gethei' as one si oup? o te : In order' to aiulyze and 
rep ort these woup s s jpaiately; you mus t have no les s tlian 7 5 s tudent s of one or more of these tliree Hispanic sub^’oup s : Mexican or 
Mexican American; Puerto Pican; or Latin American; South American; Central American; or other Hispanic or Latino.) 

YesTj 


College Course-Level Dara 

_AU participants of the College B oaid' s lutioiul SAT Validity Study must submit college course-level data Studies luve shov.-n tlut 
correc ting for course difficulty pro duces a more accurate predic tion equation for use in admis sion decisions. Submit titig cours e-level data 
requires that you pro^-ide the foUoi^ing information for each s tudent : for every^ course taken you \^iH ne?d the course abbreviation (ea ; 
TXGlOr); thelongname of thee ours e(e.^; 'English Compositionr); the grade received; the number of credits earned; and the semester' 
in which the course was taiien 


(* ^Vill b e including college c our se-lev^l data for analysis 


Colleg^oard 

inspiiing mind^ 


25 


ACES Data Submission 


• Choose from a variety of common formats including: 

-Excel 

-Access 

-SPSS 

-SAS 

-ASCII delimited format 

• Upload the file directly from a PC to ACES 

All ACES data storage and transmission are secured from end to end — that 
is, from the time it leaves the browser being used at the institution until it 
passes beyond the ETS firewall. (ETS processes ACES studies for the 
College Board.) A combination of security tools and procedures is used, 
including Secure Socket Layer (SSL), Pretty Good Privacy (PGP), and ETS 
proprietary encryption techniques. 
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Basic File for ACES Admission Validity Study 



ACES File Template - SAT Validity,' Study - 1 Horizon 

Home Insert 

Page Layout Formulas Data Review ^/iew Develop. 


Paste 




J 

Clipboard ^ 


Arial 


- 10 


A* a’ 


B I u -I m - A 

Font 







5 51 

[ 1 


Wrap Text 


Alignment 


I Merge Center 


|Ger 

I $ 




R19C11 



1 

2 

3 

4 

5 

6 

7 ^ 

^ 8 

1 

Last 

Name 

First 

Name 

Middle 

Initial 

SSN 

DOB 

Gender 

First-Year 

GPA 

HSGPAor 

Rank 

{optional} 

2 

Smith 

Lindsay 

R 

111-11-1111 

7/14/1991 

F 

3.02 

3.67 

3 

Mason 

Robert 

J 

222-22-2222 

8/19/1991 

M 

2.88 

3.45 

4 

Farher 

Lane 

C 

333-33-3333 

4/5/1991 

M 1 

2.23 

3.11 

5 










6 

Info for matching 

back to CB database 



7 



8 









9 









10 

s 

itudent info repeated down in rows 

11 


12 








Criterion (FYGPA) 

-HS Measure - optional 
because it can be 
supplied by the ACES 
system 
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Submitting the Data Fiie(s) - Login Info 


Submitting Your Data 

Prior to preparing your data for submission, re\ie\^^ the Data Preparation Guidelines for ACES Admission VaJidit>^ Studies, as as the Common Data Errors made by pre\-ious ACES users. 
Eollo’^mg these guidelines \^ill save you time and help pre^'ent errors that may delay the completion of your ACES study. 

If at any tims you havo questions about the data submission process you mai’ e-mail ACES staff for assistance. 

Ideudfym^ luformadou 

Please supply the information requested belo^^’. You should have receh'ed this in an e-mail generated by your original \’alidit>^ Study Request. You must enter this information exacdy as it appeared in 
that e-mail. The easiest way to do so is to cut and paste the information directly from the e-mail message. 

Your user account: 

Your password: 

Your request number: | 

Continue | Clear Form | 
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Submitting Data (cont.) - Course grades, Number of fiies 


Submitting Your Data (continued) 

If at any tim&you have quQSrtions about ths data SrUhmiSr^ion procoSrS. you may' Q-mail A CES staff for assisrianco. 

Course Grade luformadou 

You indicated on your request that you ’^^'ould be submitting college course -level data. Please ans\^'er the questions below speciftiug how you ’^^iU be submitting these data. 

’V^’iU grades appear in numeric or letter form? 

f* letter grades 
C numeric grades 

\Miat is ±e range of possible numeric grades, or numeric equh'alents of alphabetic grades, at your school'^ 

Lowest^ I Highest'^ | 

\Miat is the maximum number of courses per student for ’\\'hich grades ’h^iU be submitted'^ 

|- Please Select - 

Data liLe luformadou 

Ylien providing course grade data, you have the option of submitting a single file T.^ith all student data or submitting two separate files, one \^ith admission-related data, the other ’^^ith course grade 
data. Ylien submitting separate files, certain requii'ements and restrictions and file formatting rules apply, ’H^'hich you may want to re\iew. 

In how many data files (and if t\\'o, in what format) you be submitting data for this study request'^ 

submitting one file (horizontal format) 

C submitting two files (both in horizontal format) 

C submitting two files (^^ith course grade file in ^'ertical format) 


Continue | Clear All Values | 



SubmittiriQ Dsts (cont.) - Grade equivalents 


More About Your Study Request (coutmiiecl) 

yar fjw ycu havs qn£:i.iiGnz ahcusihs daia prGce:i.EryGU mav s-maH A pr as.::j::jflfKa 

Pl?a^? supply iiumaic equivalaitrf-Dralllstta'^ad?^ tliat aie u^fd in the GPA calculation. Giade^ tljaLdo notliaveanumaic value, ^ucha^ AU (audit), S (satisfactory): and P (pass) should not 
be itidudedm this table, but they should leuuiti on voui' data iile as pait of your students' records. 


Le tier Grade 


Xumerie 

Equivalenl 


10 r 


Continue | Clear all values 


( CdHegeBoaid 

inspiiing mind^ 


30 


Submitting Data (cont.) - Labeling mto 


Submitting Your Data (continued) 


If at any time you have questions about the data subtni^sio?^ process you mai' e-mail ACES staff for assistance. 

About Your Data lUe 


Please select the year that students represented m your data entered college|2008 ^ (required) 

Please indicate ’\^'hat you ’h^'ould like to call the data file you wiS. be submitting. This should be a simple descripth'e label, e.g., "Entering Class of Fall 2008." 


File label: f 


(required) 


Please specific how your student name is formatted: (required) 
f* In separate fields: first middle last (any order; e.g., Public 


Jolm 


) (recommended) 


C In a single field: last, first, middle (e.g., 
C In a single field: first, middle, last (e.g., 


Public, John Q. ) 


John Q. Public ) 


Specific your file t>pe belo\^^ (required) 

YEcrosofl: EXCEL Spreadsheet C Tab -Delimited ASCII C Fixed Length ASCII 
C Xlicrosoft ACCESS Database C SPSS Portable File 
C Comma-Delimited ASCII (CS\0 O SAS Transport File 


Continue Clear all values 
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Submitting Data (cont.) - variable locations/values 


Delimited (tab, comma) ASCII or XLS (Excel) Layout Table 



Label 

Excel CoLumu 
No. or ASCn 
Var. Fosidoo 
No. 

NDB.Value 

Max.Yalue 

First Name " 



Middle Name (Optional) 





Last Name " 





Gender ” 





Date of Birth ” 





SSN* 


1 


^^B 

Home ZIP (strongly recommended) 


1 1 



HS Code (strongly recommended) 




^^B 

Criterion * 

First- Year GPA 




Cumulative GPA (if other than First- Year GPA) ” 


1 1 


1 — 

Retentio-n Indicator 




^^B 

Major area of study (when a^'ailable) 





Unh'ersit>'- As signed StLident ID (required) " 





Add"l Predictor 1 

Matrix 

1 1 



Add"! Predictor 2 

Need 

1 i 


1 I 

Add! Predictor 3 

Filed Fafsa 




Add"l Predictor 4 

Residency 




Add"l Predictor 5 

^TJE 

1 

1 

1 

Add'l Subgroup 1 

1 Abilit^^ rating in math 

From ACES 



Add"l Subgroup 2 

|Abilit>' rating in science 

From ACES | 

HS Avg * 


1 

1 

1 : 

Course 1 Label (abbredation) " 




^^B 

Course 2 Label (abbreviation) " 





1 _ _ _ ^ 
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Obtaining the ACES Report 


ACES notifies the file subnnitter via e-mail when the 
study is completed 

The document is password protected for confidentiality 
and is encrypted until down oaded at the institution 

In addition to the ACES report, institutions receive an 

electronic copy of the student data with added 
information on: 

• students' scores for the SAT, SAT Subject Tests, Advanced 
Placennent exann scores 

• demographic information extracted from the ACES database 

• statistics generated during the processing of the study (the 
predicted FYGPA). 
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ACES Admission Vaiidity Report 


ACES 




Admission Validity 
Report for Sample 
One University 

Mt* tn tiu« rappvt m« not f prwwattiw «f aRy inatltvtson. Ml ar« 

BypotMcicM uM war* 9*e*r*c*« tor CA* fax* pvzp»** ot craaciag cat* tMpx* r«pc«rt. 

Entering Class of FaD 2007 

Youl College Board ValiditY Report b designed to assist your 
ifistftuticn in vdidaling your adnusstort decisiorts. This report 
provides a nontechnical dscussion of important findmgs. 


ADMITTEX) CLASS EVALUATION SERVICE”* 
WWW.COLLEGEBOARD.COM 
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Dwoybowor #>e Datyi tw On* tfcmwmy 

EiiiuMna nawdMladmimiAmMiMraa 


Uiii0 pmiidad Fn«-Y«*r GMfaf SilkM ttMknh 

U»n|| #« pra^Kkid FnfYMr GMIv curwrt tlMdinb la 
idMlify tiudMte paatAK *1 'Hk kx nolcaiM*totthMdi|im 
M Sample On* Uaratitii 

10 Ei«ruwwo()i*aic»<retoa»*cdicgr<w<)r>l^ 

It EiWufln9«mOma«enrmonmNMmfo(«Mlion^ 

of students « la^uBstod by Sample Ota* UnmwMty 


l>*dki»oa «QMticiM • 1b* prvdOkd Rm-VMT 6f* « UMftd 
• «raiixaul*i *pphc*nlsaml 
a cd OMiaNid uuimMs 

il»tK#i<b*< 1 ooi>«i o ><r<i* 2007 >woll > iSti>MiiS>a>pl>Qi>* 

U«n«a4v 

IM ol CN Iw afeidonto P**mMv *1 no! Ib) Rot «ampMn| Vi*n 
daixM* at Smipd* Ok* Ummi ate 
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Section 1 : Evaluating individual admission measures 


Individin I admission measiures in yourstudir 


Strang PradictDr^ 

n| 

pTEdctivie StTETigth Icorraiartion) 

HS Rank 

g70 


SATi!>jticai Reading 

970 


SAT \%'riting 

970 


Mcdaraba PrEdiotore 

SAT Math 

970 


Q.2.] 

^ AP Examt 

702 

I 

Q.E9 

SATSdjj: High-NonLang 

7BE 

’ in, D.2S 

4 EAT SubjT&tlE 

7B£ 

A'"'?;,.. ^ 

jc \ ■ '-it > 

Weak pTEdictofa 

^ Hcnora nr AP CDuraea 

7B2| ]0.22 


Other Bihnission neasi res ava ilable 


Strung Pradictora 

N 

pTEdctive Strength icDrraiation) 

SATStlsj: Math Le^ei 1 

241 


Ci.4b 

SATStij: U.S. Hi.ttcfv 

11B 


MfDdaraba PrEdiritors 




SAT Siisj: Literature j 

I “I 

Q.S1 

Weak pTEdictofa 

None arcaiiabie 


This section summarizes the predictive strength of the individuai admission measures in the study. 


The second anaiysis may inciude resuits for predictors, such as SAT Subject Te^, that institutions 
did not expiicitiy choose to study but were present in their students' records. / Colleg^oaid 

inspiiing mind^ 
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Section 2: Evaluating combined admission measures 


SAT coinbinadois 


I ^ 


Add. 

fYodclETS 




■I^dcai 

Ht-sdiig 

^'bt1 

V^flrci 

HSf^nb 

N 

PrDdktK? ul'Bijjfh ImiJdpIt EHrgl^kxiL' 


S 

91 

94 



m 




17 

7D 



m 



17 

IS 

7D 

44 

1 


It 

■la. 



SAT SAT Subject Test canbinadois 


SAT 

SATSiij#:! 

lul: 

HSHztfi: 

Add. 

fVcdEbTs 

N 

PkDdktiMi Slroic^ IniJdpJt [DmliliorLi 

OkEzJ 

Fta-adng 


fating 

High- 

riodun 

a 

27 


2S 

r ■ 


■9£ 

DL5S 

17 

19 

2'3 

1£ 



■9Z 

G£D 

IS 

m 


11 


1 

■9^ 

Q£.1 


This section combines the admission measures that were evaluated individually in Section 1 of the 
report to find the best prediction of success. 


These tables display the corrected multiple correlations between combinations of admission 
measures and the criterion. The bars at the right of each table represent this pi;ig^ctive strength 
(mu^jple correlation) for each combination. 


pr^cti 
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Section 3: Using predicted First-Year GPA for future students 


Percent ol slideils expected leeam a Hrst-\Tear GPA of al least; 



Targetsd Frst- Year GPA 

1.3 

3.0 

3.6 

3.0 

3.5 

IJO 

2J0 

9a 

5D 

5 




2.1 

97 

GZ 

ID 




22 

96 

73 

17 




22 

99 

GZ 

3E 

1 



2.a 

99 

G9 

37 

3 



2.5 

99 

9a 

5D 

5 



2JG 

99 

97 

GZ 

10 



2.7 

99 

96 

73 

17 



2ja 

99 

99 

GZ 

3G 

1 


22 

99 

99 

G9 

37 

3 


3J0 

99 

99 

9a 

9D 

5 


3.1 

99 

99 

97 

G3 

10 


3:z 

99 

99 

96 

73 

17 


3.3 

99 

99 

99 

G3 

2fi 

1 

3.J 

99 

99 

99 

G9 

37 

2 

3.5 

inn 

99 

99 

9a 

90 

9 

3JG 

lao 

99 

99 

97 

GZ 

10 

3.7 

IDO 

99 

99 


73 

17 

3J3 

IDO 

99 

99 

99 

33 

3G 

3.g 

IDO 

99 

99 

99 

39 

37 

IJO 

IDO 

IDO 

99 

22 

9a 

90 


This table can be used 
to estimate the 
likelihood that a student 
with a particular 
predicted First- Year 
GPA will earn a given 
First- Year GPA. 
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Section 4: Using the predicted First-Year GPA for current 
students to identify students possibly at risk for not completing 
their degrees at Sample One University 

Some students earned a First- Year GPA lower than that predicted by their preadmission credentials. Research has shown that these 
students a re at a higher ri sk for not compi eti ng thei r cleg rees. Thi s i nf ormation can be used to identify students who are possi bl y at 
risk for leaving Sample One University prior to graduation. 


Summary of Performance 


Number who performed 
higher than predicted 

Number who performed 
as well as predicted 

Number who performed 
lower than predicted 

Total number of students 

Percent who perfonmed 
lower than predicted 

372 

498 

108 

978 



• 10BstudentsJ1.0%ofthesample,performedsubstartiallybelciwtheirpredictedFirst-YearGPA. ;; f 

• Because these students may be more likely to drop out and may benefit from additional counseling, their student IDs are listed individually in 
Appendix C. 

• Based upon the standard deviation of the predicted F|irst-Year GPA for the entire sample, students whose actual First- Year GPA was one or more 
standard deviation(s) abme the predicted value are considered to be performing higherthan expected. Students whose actual First-Year GPA was one 
and one-half or more standard deviations be low that predicted value are considered to be perfomning Icrwer than e.^pected. and the rest are considered 
to be performing as well as expected. 


Some students earned an FYGPA lower than that predicted by their 
preadmission credentials. Research has shown that these students are at 
a higher risk for not completing their degrees. This information can be used 
to identify students who are possibly at risk for leaving Samo^One 
Ig^iversity prior to graduation. L CollegeBoard 
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Section 5: Evaluating predictions for specific groups of students 


Equations computed for all students may not accurately reflect the performance for some subgroups of students who attend your 
institution. For this reason, ACES compares predicted First-Year G PA with actual First-Year GPAto check for significant differences 
and identifies the groups of students whose actual performance in college is higher or lower than predicted. There are many 
possible reasons for the differences in performance between groups, and there is no agreed-upon remedy. 


First- Year G PA for Specific Groupincjs of Students 

Category 

Groups 

Nuniber of Students 

Average First- Year GPA 

Predicted 

Actual 

Difference 

Gender 







Males 

450 

2.88 

2.80 

0.08 


Females 

528 

2.97 

3.02 

-0.05 

Residency 







Dut-of-State 

428 

2.94 

2.90 

0.04 


In-State 

550 

2.93 

2.99 

-0.00 


•This table shows over- and under-prediction by subgroup. 
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Section 6: Evaluating combined admission measures for additional 
groups of students as requested by Sample One University 


Residency — Out-of-State 


SAT combinations 


S4T 

HSRai^ 

JUd. 

FYeddETs 

hi 

Predciwt Sirsigdi ijnuHiile oomliliorLi 

D-hk^l 

Ffcidng 

Muh 

Wrtlirg 

31 

21 




m 


IS 

le 

19 



42e 


IS 

le 

19 

U 

1 

42E 

Dl?1S 


SAT Sl sat Subject Test combinations 





Residency — In-State 


This section includes 
the corrected multiple 
correlations for different 
subgroups examined in 
the study. 


SAT combinations 


SAT 


Add. 

PredclETS 

hi 

Predciwt Sirsigdi ijnufliile oomliliorLi 

D-hk^l 

Fbdng 

U^h 

lA-rrlirg 

30 

51 

29 




Iolej] 

21 

2D 

29 




DJ9S 

30 

21 “ 

IT 

26 

b 


[LE5 


SAT Sl sat Subject Test combinations 


SAT 

SAT 5uh^ 
Test: 

H5Dari( 

Add 

Pmdictori- 

FJ 

FYedkdw 5!r<rgti ijiuh^t- [urdatiail 

Cfilical 

l^adhg 

Uadi 

fMidng 

Higly 

hknLang 

2D 

3S 

£9 

23 



4EJ] 

DJS1 

ID 

30 

19 

21 

31 


AEJ] 


ID 

IS 

H 

U 

31 

1 

4EJ] 

[197 
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Appendix A: Prediction equations 

Several equations are given so that institutions are provided with the information needed to 
choose the best combination of predictors for each st udent. 


Api>eiidlx A: Precllctlon equations 

The numbers in thetableis) in this section describe the prediction equations devElopedfor Sampli 
represents: 1) a model with a different set of predictors used to formulate an equation tor use in 
potential students \vtiose records contain the variables cTiosen for this study, and 2) the comespom 
these predictors. 

The first four ro\vs of tl^ table show 

* The number of student records used in that analysis 

* The resulting multiple correlation 

* The multiple con-elation adjusted for the rostriction in the rarTge of scores for this group of studen 

* The standard error of the prediction equation 

The remaining mm in each column display the raw regression weights to be applied to known prefiii 
predicting First- Year G PA. 

Your decision on which equation to use is based on the information available in each student’s record i 7 ^ , 
institution or state. For example, if a student has an SAT Critical Reading score of 550, an SAT Math sci | 

V\titing score of 550, and supplies no other information, the appropriate prediction equation (using dal^ 1^ — 

Predicnd First-Year GP.T = - (SAT x .1:11 CR wetfht) - (SAT M score 1 

^ T W score SAT JV weight) \ 

Predicted First-Tear GPA = 1.405640 - {550 x 0.001250/ - r59€ x 0.000520} ^ (550 x 0.0^ | 

V\Tien using a student's rank in class in your prediction equation, you must first convien that rank to tl^ sa| 1 
process your study. The formula to do this is: 

nOO X {I divided rfie number of jn^c^ertfj in the class}). 

For exa mpl e . if a student s rank in ci ass is 20th out of a class of 74 students you wou Id use the for m ula to dl ^ this 

intl^ prediction equation. 

High School Rank = a 00 [I - {20 - 74)}j = 73 
ACES creates prediction equations wtien tfere are 75 or more students within a group. 
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Using the prediction equations in 
Appendix A... 

• First obtain a Predicted First Year GPA 

• Example SAT CR, SAT M, SAT W, HS Rank 

• 550,550,530,85 yields PFYA = 2.90 

• Find row with PFYA of 2.90 

• We can see that 89% of the students with a PFYA of 2.90 
are expected to earn 2.5 or greater. 

• Using the PPG (Predicted Performance Calculator) 
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Predicted Performance Calculator 



Tool delivered 
with each 
ACES 
admission 
validity study - 
specific to your 

institution. 
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Circling back to the probability table from Section 3 


Percent of students expecled lo cam a First- Year GPA cl al Icbsi; 



T 


GW 





1.5 



3.D 

3.S 

4.0 



94 

5il 

& 





Z.1 

9T 

K 

15 







73 

17 






99 

8^ 

2ft 

1 



p 

2.4 

99 

Be 

37 

2 



r 


99 

54 

aj 

5 



d 

2.e 

99 

97 

6? 

1 

ID 

1 



t 

i 

3.7 

53 

96 

73 

1 " 



1 

e 

d 

F 

1 

2.fl 

99 

59 

92 

2D 

1 



99 

Be 

6» 

37 

1 


3-C 

99 

m 

54. 

56 

5 


r 

1 

3.1 

99 

35 

57 

D2 

1C 


t 

Y 


99 

99 

96 

73 

17 


e 

a 


99 

35 

35 

B2 

2« 

1 

D 

r 

34 

99 

W 

95 

B9 

37 

1 

G 

p 

3.5 

100 

35 

55 

34 

SO 

S 

4 

3.C 

lOO 

99 


97 

62 

10 


3.7 

100 

95 

95 

3D 

73 

17 


3.B 

100 

95 

95 

BB 

B2 

2B 


3.-g 

100 

95 

55 

99 

19 

37 


4.fl 

100 

135 

95 

39 

54 

SO 


You can see that 
this student has an 
89% probability of 
achieving a 2.5 or 
higher in his/her 
first year of college 
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Appendix B: Statistical summaries of study variables 



Ap[>eiidlx B: StaUstLoal summaries of study variables 



Avtrago Scorti ByGfl-idtr 



Totil 

Hill 

Fenile 


N 

Mean 

SO 

N 

Mean 

SO 

N 

Mean 

SD 

First-Year GFft 

97B 

2.93 

0.B3 

m 

2.BD 

Q6B 

52B 

3.EE 

0.93 

SAT Critical Reading 

97B 

56B 

7B 

m 

576 

74 

52B 

562 

74 

SAT Math 

97B 

B72 

7B 

493 

595 

75 

52B 

55B 

73 

SATWritir^ 

97B 

&33 

31 

493 

5B6 

73 

52B 

99B 

33 

HSRank 

97B 

7B.EQ 

15.35 

493 

7Z37 

15.73 

52B 

03.13 

14.23 

SAT Sdij: High-NonLang 

m 

B31 

70 

303 

531 

74 

432 

553 

65 

# Honors or APeoursas 


1.4B 

1.B9 

303 

1.3D 

1.37 

432 

1.B0 

1.93 

# AP Exan^s 

732 

0.9B 

1.33 

303 

D.B6 

1.24 

432 

1.03 

1.39 

# SAT Subj Tests 

732 

0.97 

1.43 

303 

D.91 

1.43 

432 

1.01 

1.44 

SAT Sdjj: U.S. History 

11B 

563 

91 







SAT L iteratura 

BD 

BD9 

31 







SAT Stisj: Math Level 1 

241 

57B 

72 

93 

593 

71 

14B 

563 

70 



Gorr^laitniis Prodicltr^ aid Cif?:i-Yv ir GPA 



Tetal 

Male 

Female 


N 

R 

Fiiadj’' 

N 1 

1 R 

Rtadjl 

N 

R 

Riadj) 

SAT Critical Reading 

97B 

0:Z4 

OiZ 

493 

D.15 

Ct33 

52B 

0.34 

0.54 

SAT Math 

97B 

0.1 B 

0.39 

493 

D.13 

Ct33 

52B 

0.32 

0.53 

SAT Writing 

97B 

0:Z7 

0.42 

493 

Q.1D 

Ct2B 

52B 

0.37 

0.53 

HSRank 

97B 

0.41 

0.52 

493 

D.4D 

Q49 

52B 

0.3B 

0.53 

SAT Stisj: High-NonLang 

732 

OJO 

0.4J3 

303 

D.13 

DL35 

432 

0.30 

0.55 

# Honors or APooursas 

732 

0.10 

0.22 

303 

D.07 

Q19 

432 

0.10 

0.25 

# AP Exams 

732 

o:Z5 

0.39 

303 

D.26 

0.3B 

432 

0.24 

0.42 

# SAT Subj Tests 

732 

o:zo 

0.36 

303 

D.2D 

Ct32 

432 

0.19 

0.4J3 

SAT S^h\: U.S. History 

11B 

o:zi 

0.41 







SAT Stinj: L iteratura 

BD 

o.u 

0.31 







SAT Stisj: Math Level 1 

241 

OJO 

0.45 

93 

D.23 

Q39 

14B 

0.40 

0.55 





This appendix 
inciudes summary 
statistics and 
adjusted (for range 
restriction) and 
unadjusted 
correiations for 
predictors and 
criterion by totai 
group and gender. 
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Appendix C: List of iDs for students possibly at risk for not 


To help you target retention efforts at Sample One University. ACES has listed those students whose actual academic performance 
fell one and one-half or more standard deviations below their predicted First- Year GPA. The standard deviation of tl^ predicted 
First-Year GPA for all students in tl^ study was used to identify these at-risk students. 

Notes: 

* These differences are based on the equations devo loped for all students and do not reflect any analyses for specific demographic 
groups. 

* Students who earned a First-Year GP.A of less than 2.0 are not shown in .Appendix C. as these students are readily identified as 
being at academic risk. 

* You did not submit course grades, so First-Year GPA is unadjusted for course- to-course differences in grading standards at 
Sample One University. 


StudantID 

F rat-YBar GPA 

Gandar 

R.-E 

E9L 


FVEcJictad 

Actual 

Oiffei-ncB 


S.3D 

2.02 

-1.33 

M 

W 

Y 

XW-H-JOOOl 1 

3.2^ 

2.15 

-1.C9 

F 

W 

Y 


3.12 

2.ra 

-l.CQ 

F 

W 

Y 

XW-JK-JOOOl 

3.03 

2.CO 

-1.C0 

F 

w 

Y 

XW-5H-5WXX 1 

3.1D 

2.04 

-1.06 

F 


Y 


3.06 

2.C6 

-l.CO 

M 

w 

Y 


3.33 

2.33 

-l.CO 

F 

w 

Y 

JCXX-XX->DCOt 1 

3.30 

2.43 

-0.97 

F 

w 

Y 

XW-H-JOOOl 

2.93 

2.CG 

J3.96 

F 

w 

Y 


3.02 1 

[ 2.06 

J3.96 

F 

w 

Y 

JCXX-XX-XMX 

2.99 

2.07 

-0.92 

M 

B 

Y 


3.02 

2.13 

J3.B9 

F 


Y 


3.4? 

2.59 

J3.E3 

M 

W 

Y 

XW-5K-500LX { 

3.02 

2.13 

J3.B4 

F 

W 

Y 

JCXX-M-XXXX 

2.33 

2.00 

-0.B3 

F 

W 

Y 

JCXX-WC-XXM. 

3.25 

2.44 

-0.B1 

F 

■iV 

Y 

X»t-5W-KfTC( 

3.19 

2.33 

-0.B1 

M 

w 

Y 

JCXX-XX-XMX 

2.94 

2.15 

-0.79 

F 

w 

Y 

XXX-M-XXXX 

3.13 

2.35 

-0.73 

F 

w 

Y 

JCXX-XX-XXM 

3.19 

2.43 

-0.76 

F 

w 

Y 

XW-5K-500LX 

3.01 

2.36 

J3.75 

F 

w 

Y 

xw-m-jowx 

2.39 

2.15 

J3.74 

M 



JCXX-KX-XXXX 

2.39 

2.17 

-0.72 

M 

w 

Y 

XW-H-JOUX i 

3.23 

2.57 

J3.71 

F 



XW-H-JOOOl 

2.® 

2.39 

J].7a 

M 

w 

Y 

JCXX-WC-XXM 

3.05 

2.35 

-0.70 

M 

w 

Y 

JCXX-XX-XXM 1 

3.03 

2.42 

J3.D6 

F 

■iV 

Y 




This appendix helps institutions to 
target retention efforts. A list of IDs 
for students possibly at risk for 
dropping out or transferring is 
provided here based on when 
students’ actual academic 
performance fell one and one-half 
or more standard deviations 
below their predicted First- Year 
GPA. 

Students with a FYGPA of less 
than 2.0 are not shown as these 
students are readily identified as 
being at academic risk 
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Section IV 


ACES Placement Validity Studies 
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Placement Validity Studies 


You can also use ACES to conduct a predictive 
placennent validity study using scores from: 

• SAT- Critical Reading, Writing, and Math 

• SAT Subject Tests 
. ACCUPLACER 

to predict success in a college course. 

Answering questions like, "What is the appropriate cut score 
on SAT Math to place students into Math 101 versus 
remedial Math 100 at my institution?" 


48 
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Requesting a Placement Study 



• You may analyze up to five 
separate courses in your 
placement validity study. 


• There must be at least 30 student 
records for a placement validity study to 
run (or more, depending on the number 
of predictors used and the type of 
placement study requested). 
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Requesting a Placement Study (cont.) 


• You do not have to submit an additional data file for a 
placement study if you have submitted a 
comprehensive file for an admission validity study. 

• For each course, you will begin by specifying the 
tests and the predictors you are most interested in 
evaluating. 

• You may use a single predictor, or up to five predictors, for a single 
course. 
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Screenshots from an ACES SAT 
Placement Study Request: 


n 


CoUeg^oard 

Admitted Class Evaluation Service 


Placeiiieut Valiclitj' Study Request (coutiuued - 1 of 5) 

Use this fomi to select at least one SAT Test or SAT Subject Test to be used in the analysis for this course. ACES staff t.vill nutch your student data to the ACES database and extract the SAT scores for you. You ca: 
submit data for other measures of student abdity tliat you\^-ish to study. In total, you may stJadH^ up to tlve pralictor measures, but you must mclude at le^st one SAT Test predictor for each course you want to aiu 

Specth’ a Course 

The criterion for a Placement ’v'aliditv Study is a n^ade in a specific course. < Enter the name of the course 

Please specify the college course name and or number exactly as it is to appear' in the report: 

I I (required] (\imil of 2<] characters) 


Specif)- SAT Tesr Predicrors and SAT Subject Test Predictors 

Select the SAT Test predictor(s) and or SAT Subject Test predictor(s) to analyze for the course specified above. 


SAT: 

□ Critical Reading □ Writing 

□ Math 




En^Lish 

Histoiy 

Math 

Science 

Lan^ua^e 


Tests 

Tests 

Tests 

Tests 

Tests 

SAT Subject: 


□ u. S. 

□ MillrL^vel 1 

□ 3io1e^- 

□ Cll’2=Li =.l:o.- 

□ P:iy=-ic=. 

□ v-i Li Elicjc-i 

O v.-i til Li E l'irir.E 

O F:^.C.\ iri =-i c-:d 


< Choose the test(s) 

of interest 
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Screenshots from an ACES SAT 
Placement Study Request: 


You may also as many as chree additional tests or specific measures, for which you will be submitting data, to 

use as predictors in the placement validiiy study. If you intend to utilize this option, please indicate the labels for 
such measures in the boxes below. 





3 > 



(limit of 20 tei- 5 ) (limit of 20 dmac t en) (limit of 20 tei 5) 


You may include up to five separate courses in one Placement Validin- Request. 
Do vou want to include another course for analvsis usins SAT scores? 


Yes 


Oi'. Submit Completed Request if you're done. 




Save & Exit 
Clear Page ] 
bdt 1 


Save the information that has been entered up to this point and exit the browser. This will allow you to 

Clear all fields on this pa^e. 

Close the browser without savins your information. 


Any additional 
predictors? 


Additional 
courses you 
, would like to 
analyze? 
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ACES Placement Validity Report 


• The predictive validity study provides two 
probability tables for cut scores, among other 
information: 

• For a course grade of B or higher 

• For a course grade of C or higher 
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Example of ACES Placement Chart 


Cut Scores Associated with Predicted Probability of Success Criterion: 
Final Course Grade of C or Higher in EngtOO Using SAT Scores 


Cm Scor»» Ai$odti»d wiiii mici«i ProMbiiitY of Sio»*s 
Crhorlen: Rnil Court* Grodo of C or Hlghor li EnglOO 
IMng SAT Scoioi 


Tb« 1pllaMnngintO«ri4unNu9«dto<olci4ac*t^ ^ - /wn b tfvt (jbit 


MDdtl Hmtm 1 ■ 4.23677 # (0 OOSffit r SAT Dicitil K tding f , DiXETSi x SAT WNtitf 






PntMbikrv 

OfSuCCOM 

SATCrkicalfWiilMgOrtr 

$klWfiiiiftey 

C«rp«titi fttdoor 

0.06 



291 

0.90 

791 


220 

0.86 

692 

760 

173 

0.80 

621 


140 

0.75 

Cwe>- 

570 

no 

0.70 

192 

512 

006 


u\ 

476 

QIS 

060 

390 

116 

Oil 

056 

346 

370 

020 

050 

300 

3 Z 7 

000 

046 

266 

268 

020 

040 

211 

227 

^41 

0.36 



062 

030 



066 

0.25 



• 1.10 

0.20 



-140 

015 



-173 

0.10 

j 

1 

•220 

0.06 

1 

•294 


Here you can see 
that an SAT CR 
score of 556, for 
example, is 
associated with a 
75% probability of 
obtaining a C or 
Higher in ENG 100 
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Chart with Correlations and % Correctly Placed 


•Individual predictors examined (SAT CR alone, SAT W alone) 
•Composite predictor 


Logistic Bisarlal Coirelations* ol Prodiclors wilh Siccoss oii ti» Criterion 
Crilorioni Final Course Grade ol Cer Higher in EnglOO 
Using SAT Scores 
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ACES Validity Handbook 

(http://professionals.collegeboard.com/higher-ed/validity/aces/handbook) 


• It is designed to serve as a 
general reference for validity 
and includes information about 
validity beyond what is 
specifically applicable to ACES. 


• It includes specific information 
about the types of validity 
studies and their design that are 
available through the ACES 
system and helps interpret 
ACES study results. 


Validity 

ACES*- 



Admisa«n VahdCy Study 
Placement Vabdly Study 
Validity Handbook 
Teat VaidCy 
Vabdty EvOence 
Glossary 

Which Study is Right for You 


About Admtsson Valdty 
Studies 

About Placement VahdCy Studes 

Predictfve Placement ValdSy 
Studies 

Concurrent Placement Vaidty 
Studies 


Existrg Placement Program 
Evaluation 


Avoidtfig Potential Problems 
FAQ 
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Section V 

The National SAT Validity Study 

http://professionals.colleaeboard.com/data- 

reports-research/cb/recruiting 


Homepage Home > Data, Reports & Research 

> College Board Research > Recruiting for SAT Validity Study 



Data, Reports & Research 

Recruiting Four-Year Institutions for the National 
SAT Validity Study 

CUSTOMIZED ENTRY PAGES 


Higher Ed Trends & Related Reports + 

View information and tools for each of 
these professional roles: 


Online Score Reports 

1 ^ Print Article S Email Article 


|k- 12 Teacher [ v] 


SAT® Data & Reports + 




s 


PSAT/NMSQT® Data & Reports 

The College Board continues to examine the validity, fairness and effectiveness of 


AP© Data & Reports 



the SAT across a national range of institutions and students. 



College Board Research 
View All Research by Title 
AP 

CLEP® 

SAT Reasoning Test*" 
SAT Subject Tests*" 

Over 190 four-year colleges and universities have participated in this ongoing effort 
to research test validity and college success. Each participating institution receives: 

• A stipend for the work involved in assembling and submitting a data file with 
particular student information related to college performance. 

f QUICK LINKS 


Statistical Definitions 


RELATED DOWNLOADS 

- 

KbAI/NMbUI 

SpringBoard® 

Students with Disabilities 
Race, Ethnicity & Socioeconomic 
Status 

Higher Education 
Conference Presentations 

* Unique institutional admission (and placement , if desired) validity studies 
customized and designed to your specifications. 

Guidelines for the Release of Data 
(pdf/1 30.69K) 


• A comprehensive data file returned with supplementary student-level 
variables from a College Board database. 

Guidelines on the Uses of College Board Test 
Scores and Related Data (pdf/808.87K) 

• Copies of the national SAT Validity Study {.pdf/1 19KB) and other research 
studies that report data aggregated across institutions. 

^ Requires Adobe® Reader® 


Validity 

Request for Data 

Recruiting for SAT Validity Study 

The study will focus on the first-time, first-year students who began at your institution 
in fall 2010. It has been designed to include those four-year institutions with at least 

f RELATED LINKS 


AP Central® for Teachers 



250 first-year students and more than 75 SAT takers. 

The data files for the study are due by October 31,2011. 

Get additional information on the National SAT Validity Study (.pdf/68KB). 

APforK-12 
AP and Higher Ed 
The AP Press Room 
Free AP Publications 



If your institution is interested in participating in this national research project, 
please contact Sarah Arsenault at sarsenault@collegeboard.org or 212-713-8082. 
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National SAT Validity Study 


Responsible test developers and publishers must be able to 
demonstrate that it is possible to use the sample of behaviors 
measured by a test to make valid inferences about an 
examinee's ability to perform tasks that represent the larger 
domain of interest, (e.g. SAT predicts FYGPA) 

We conduct: 

• Cross-institutional, longitudinal validity and higher 
education research to inform ways to ensure that 
students are ready for and successful in college. 

• Data supplied by four-year institutions from around the 
U.S. and matched to College Board data. 
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National SAT Validity Study (cont.) 


• The ACES system is used to generate institutional validity 
reports while College Board researchers aggregate data 
files from institutions (with at least 250 first year, first 
time students) to provide national, comprehensive validity 
studies - available at www.colleaeboard.com/research 

• Each participating institution receives: 

• a unique ACES admission validity study 

• a comprehensive data file returned with supplementary student- 
level variables from a College Board database 

• a stipend for their work to create the data file 

• opportunities to learn about results before general pu' 
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Variables for National SAT Validity Study 

ACES Studies (deadline of 10/31/11) 


First-Year Data on Fall 2010 Cohort (first-time, first-year students that 
began at your institution in fall 2010) 

• Name 

• Social security number 

• Date of birth 

• Gender 

• University-assigned student ID 

• Retention to the second year ("yes" or "no") 

• First-year GPA 

• Grades in first-year courses 

• Course abbreviations for first-year courses (e.g., ENG 101) 

• Course long names for first-year courses (e.g., Introductory English) 

• Credit hours attempted for each course 

• Semester each course was taken 

• High School GPA (can be supplied by the ACES system or your 
institution) 
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SAT Validity Study results - snapshot 


• Admission Validity Study 

SAMPLE (2008 entering cohort) 

129 colleges participating in Validity 
Study (N = 246,652) 

• Schools provided first year performance data 
for Fall 2008 cohort through the Admitted 
Class Evaluation Service’’’^ (ACES portal 

Restrict sample to students who 
completed the new SAT, submitted 
self reported HSGPA, and had a valid 
FYGPA(N= 173,963) 
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Admission Vaiidity Resuits (1 of 2) 


• SAT Writing has the highest correlation with 
FYGPA among the three individual SAT sections 
(Adj. r= 0.52). 

• SAT CR (Adj. r= 0.48); SAT M (Adj. r= 0.48) 


• As expected, the best combination of predictors of 
FYGPA is FISGPAand SAT scores (Adj. r=0.63), 
reinforcing the recommendation that colleges use 
both HSGPA and SAT scores to make the best 
predictions of student success. 
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Admission Vaiidity Resuits (2 of 2) 


• The adjusted correlation of HSGPA and FYGPA is 0.56; 
the multiple correlation of the SAT (CR, M, and W 
combined) with FYGPA (Adj. r= 0.54). 

• The increment in predictive validity attributable to the SAT 
when HSGPA is taken into account is 0.07. 

• The increment in validity attributable to the Writing 
section over and above the CR and M sections is 0.02. 
When HSGPA is also considered, the increment in validity 
attributable to the Writing section is 0.01 . 


63 



CoHegeBoard 


inspiiing mind^ 


FYGPA 


Mean FYGPA by SAT Score Band, Controlling for HSGPA 
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Questions? 


Thank you! 

Feel free to email with questions or your 
interest in participating in the national SAT 
Validity Study at eshaw@colleqeboard.org. 


Researchers are encouraged to freely express their professional judgment. 
Therefore, points of view or opinions stated in College Board presentations do not 
necessarily represent official College Board position or policy. 
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